Two-Layer Generalization Analysis for Ranking Using Rademacher Average
نویسندگان
چکیده
This paper is concerned with the generalization analysis on learning to rank for information retrieval (IR). In IR, data are hierarchically organized, i.e., consisting of queries and documents. Previous generalization analysis for ranking, however, has not fully considered this structure, and cannot explain how the simultaneous change of query number and document number in the training data will affect the performance of the learned ranking model. In this paper, we propose performing generalization analysis under the assumption of two-layer sampling, i.e., the i.i.d. sampling of queries and the conditional i.i.d sampling of documents per query. Such a sampling can better describe the generation mechanism of real data, and the corresponding generalization analysis can better explain the real behaviors of learning to rank algorithms. However, it is challenging to perform such analysis, because the documents associated with different queries are not identically distributed, and the documents associated with the same query become no longer independent after represented by features extracted from query-document matching. To tackle the challenge, we decompose the expected risk according to the two layers, and make use of the new concept of two-layer Rademacher average. The generalization bounds we obtained are quite intuitive and are in accordance with previous empirical studies on the performances of ranking algorithms.
منابع مشابه
Generalization Analysis of Listwise Learning-to-Rank Algorithms Using Rademacher Average
This paper presents theoretical analysis on the generalization ability of listwise learning-to-rank algorithms using Rademacher Average. The paper first proposes a theoretical framework for ranking and then proves a theorem which gives a generalization bound to a listwise ranking algorithm based on Rademacher Average of the class of compound functions operating on the corresponding listwise los...
متن کاملGeneralization Bounds for Time Series Prediction with Non-stationary Processes
This paper presents the first generalization bounds for time series prediction with a non-stationary mixing stochastic process. We prove Rademacher complexity learning bounds for both average-path generalization with non-stationary β-mixing processes and path-dependent generalization with non-stationary φ-mixing processes. Our guarantees are expressed in terms of βor φ-mixing coefficients and a...
متن کاملGraph-based Generalization Bounds for Learning Binary Relations
We investigate the generalizability of learned binary relations: functions that map pairs of instances to a logical indicator. This problem has application in numerous areas of machine learning, such as ranking, entity resolution and link prediction. Our learning framework incorporates an example labeler that, given a sequenceX of n instances and a desired training size m, subsamples m pairs fr...
متن کاملBounds for Learning the Kernel: Rademacher Chaos Complexity
In this paper we develop a novel probabilistic generalization bound for regularized kernel learning algorithms. First, we show that generalization analysis of kernel learning algorithms reduces to investigation of the suprema of homogeneous Rademacher chaos process of order two over candidate kernels, which we refer to it as Rademacher chaos complexity. Our new methodology is based on the princ...
متن کاملGeneralization Bounds for Learning the Kernel -
In this paper we develop a novel probabilistic generalization bound for learning the kernel problem. First, we show that the generalization analysis of the kernel learning algorithms reduces to investigation of the suprema of the Rademacher chaos process of order two over candidate kernels, which we refer to as Rademacher chaos complexity. Next, we show how to estimate the empirical Rademacher ...
متن کامل